The advent of the Internet and the capability of accessing large local and widely distributed sources of information have facilitated increasing attention for ways in which to search or mine such information. The Internet may be viewed as a gigantic data repository with tremendous redundancy that can be exploited for information retrieval. Thus typical data retrieval systems retrieve large numbers of documents relevant or even irrelevant to a search query.
Automatic question answering is a very useful sub-problem associated with information retrieval. A goal of a question-answering system is to provide a concise answer to a natural language question. Instead of deluging the user with a long list of documents possibly containing relevant information buried within, automatic question answering attempts to directly answer the user question by extracting the answer from one or more documents rather than returning a collection of documents that may contain relevant or even irrelevant information. Most conventional systems are designed to perform question answering by using extensive knowledge engineering in the form of handwritten patterns which contain the answer given template that matches a question, or very detailed natural language processing, or a combination of both. Automatic question answering typically works by first locating a number of text passages from a (typically very large) text collection that contain potential answers, and then extracting the answers from this collection of passages.
Most automatic question answering systems work by defining a set of question types, building recognizers that automatically assign a question to one or more question types, and then using hints based on the question type and the question itself for finding answers to questions of that type. For example, there may be a question type: <Person's Birth Date> to which question of the form: “When was . . . born?” is assigned. An answer hint for this question type might be that given a question “When was X born?”, if the system finds a string “X was born on <DATE>”, then <DATE> is likely to be the answer to the question.
Generally, given a question Q and some text T, it is desired to assess the probability that T contains an answer to Q. If Q=“Who is Bill Gates married to?”, and T contains the string “ . . . Bill Gates is married to . . . ”, then T is likely to contain an answer. A case like this can be solved by query rewrites, that is, given a question, search for strings that are part of a declarative sentence answering the question. For example, “Where is the Louvre?”→“the Louvre is located in”; and “Who was the third American president?”→“the third American president was”.
However, finding answers by rewriting a question into a partial declarative answer only partly solves the problem. Often, answers appear in a less straightforward relation to the question. For example, the sentence: “Bill and Melinda Gates have three children” contains the answer to the question “Who is Bill Gates married to”, although the relation between question and answer string is a more subtle.
A deficiency in conventional systems is that there is no learning capability. The system handles the nth question of some type in the same way as the question would be handled if processed first. Thus a question that when rewritten provides little or no straightforward relationship to a potential answer poses a significant problem to conventional systems.
What is needed is a system that can process through such non-straightforward queries and learn their subtle nuances such that when given a question of similar type, the system would use those now known terms to improve accuracy on the new question.